Pandas Profiling Report

Dataset statistics

Number of variables	3
Number of observations	1000
Missing cells	0
Missing cells (%)	0.0%
Duplicate rows	0
Duplicate rows (%)	0.0%
Total size in memory	23.6 KiB
Average record size in memory	24.1 B

Variable types

Numeric	2
Categorical	1

Alerts

`address` has a high cardinality: 968 distinct values	High cardinality
`address` is uniformly distributed	Uniform
`df_index` has unique values	Unique
`statementID` has unique values	Unique

Reproduction

Analysis started	2022-06-01 21:25:32.435117
Analysis finished	2022-06-01 21:25:49.435177
Duration	17 seconds
Software version	pandas-profiling v3.2.0
Download configuration	config.json

df_index
Real number (ℝ_≥0)

UNIQUE

Distinct	1000
Distinct (%)	100.0%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	6728.444

Minimum	34
Maximum	13225
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	7.9 KiB

Quantile statistics

Minimum	34
5-th percentile	705.75
Q1	3291.5
median	6828
Q3	10201
95-th percentile	12602.5
Maximum	13225
Range	13191
Interquartile range (IQR)	6909.5

Descriptive statistics

Standard deviation	3883.28384
Coefficient of variation (CV)	0.5771444096
Kurtosis	-1.256303895
Mean	6728.444
Median Absolute Deviation (MAD)	3442.5
Skewness	-0.03639327311
Sum	6728444
Variance	15079893.38
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
9699	1	0.1%
909	1	0.1%
12953	1	0.1%
1926	1	0.1%
1354	1	0.1%
11599	1	0.1%
9518	1	0.1%
1158	1	0.1%
9550	1	0.1%
2183	1	0.1%
Other values (990)	990	99.0%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
34	1	0.1%
36	1	0.1%
46	1	0.1%
50	1	0.1%
60	1	0.1%
73	1	0.1%
77	1	0.1%
104	1	0.1%
127	1	0.1%
141	1	0.1%

Value	Count	Frequency (%)
13225	1	0.1%
13219	1	0.1%
13212	1	0.1%
13211	1	0.1%
13202	1	0.1%
13199	1	0.1%
13192	1	0.1%
13172	1	0.1%
13154	1	0.1%
13150	1	0.1%

statementID
Real number (ℝ_≥0)

UNIQUE

Distinct	1000
Distinct (%)	100.0%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	9.338909544 × 10¹⁸

Minimum	4.805416525 × 10¹⁶
Maximum	1.842633337 × 10¹⁹
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	7.9 KiB

Quantile statistics

Minimum	4.805416525 × 10¹⁶
5-th percentile	9.011215325 × 10¹⁷
Q1	4.802867636 × 10¹⁸
median	9.474764251 × 10¹⁸
Q3	1.404373252 × 10¹⁹
95-th percentile	1.74850693 × 10¹⁹
Maximum	1.842633337 × 10¹⁹
Range	1.83782792 × 10¹⁹
Interquartile range (IQR)	9.240864884 × 10¹⁸

Descriptive statistics

Standard deviation	5.317991302 × 10¹⁸
Coefficient of variation (CV)	0.5694445671
Kurtosis	-1.215034819
Mean	9.338909544 × 10¹⁸
Median Absolute Deviation (MAD)	4.607376413 × 10¹⁸
Skewness	-0.05138643122
Sum	9.338909544 × 10²¹
Variance	2.828103149 × 10³⁷
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
9.120979902 × 10¹⁸	1	0.1%
1.288311176 × 10¹⁹	1	0.1%
7.693078692 × 10¹⁸	1	0.1%
1.956796832 × 10¹⁸	1	0.1%
5.222305948 × 10¹⁸	1	0.1%
1.113272739 × 10¹⁹	1	0.1%
7.511874901 × 10¹⁷	1	0.1%
1.082068867 × 10¹⁹	1	0.1%
9.418428509 × 10¹⁸	1	0.1%
8.549520696 × 10¹⁸	1	0.1%
Other values (990)	990	99.0%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
4.805416525 × 10¹⁶	1	0.1%
5.683279896 × 10¹⁶	1	0.1%
6.903824493 × 10¹⁶	1	0.1%
7.63257287 × 10¹⁶	1	0.1%
7.791625436 × 10¹⁶	1	0.1%
9.287754387 × 10¹⁶	1	0.1%
1.116827498 × 10¹⁷	1	0.1%
1.126026089 × 10¹⁷	1	0.1%
1.389205943 × 10¹⁷	1	0.1%
1.419158908 × 10¹⁷	1	0.1%

Value	Count	Frequency (%)
1.842633337 × 10¹⁹	1	0.1%
1.842188255 × 10¹⁹	1	0.1%
1.839321107 × 10¹⁹	1	0.1%
1.838407239 × 10¹⁹	1	0.1%
1.837605231 × 10¹⁹	1	0.1%
1.836613845 × 10¹⁹	1	0.1%
1.835866244 × 10¹⁹	1	0.1%
1.833474244 × 10¹⁹	1	0.1%
1.832524726 × 10¹⁹	1	0.1%
1.831628691 × 10¹⁹	1	0.1%

address
Categorical

HIGH CARDINALITY
UNIFORM

Distinct	968
Distinct (%)	96.8%
Missing	0
Missing (%)	0.0%
Memory size	7.9 KiB

Kemp House, 160 City Road, London, EC1V 2NX	7
20-22 Wenlock Road, London, N1 7GU	6
20-22, Wenlock Road, London, N1 7GU	6
71-75 Shelton Street, Covent Garden, London, WC2H 9JQ	4
71-75 Shelton Street, London, Greater London, WC2H 9JQ	4
Other values (963)	973

Length

Max length	102
Median length	77
Mean length	50.695
Min length	26

Characters and Unicode

Total characters	50695
Distinct characters	74
Distinct categories	8 ?
Distinct scripts	2 ?
Distinct blocks	1 ?

The Unicode Standard assigns character properties to each code point, which can be used to analyse textual variables.

Unique

Unique	955 ?
Unique (%)	95.5%

Sample

1st row	113, Ellisland, Kirkintilloch, Scotland, G66 2UA
2nd row	2, Queens Quay, Belfast, Co Antrim, BT3 9QQ
3rd row	Redbridge Farm, Dolmans Hill, Lytchett Matravers, Poole, Dorset, BH16 6HP
4th row	Churton House Chester Road, Churton, Chester, CH3 6LA
5th row	122 Sunnymead, Peterborough, PE4 5BZ

Common Values

Value	Count	Frequency (%)
Kemp House, 160 City Road, London, EC1V 2NX	7	0.7%
20-22 Wenlock Road, London, N1 7GU	6	0.6%
20-22, Wenlock Road, London, N1 7GU	6	0.6%
71-75 Shelton Street, Covent Garden, London, WC2H 9JQ	4	0.4%
71-75 Shelton Street, London, Greater London, WC2H 9JQ	4	0.4%
Chase Business Centre, 39-41 Chase Side, London, N14 5BP	3	0.3%
27, Old Gloucester Street, London, WC1N 3AX	3	0.3%
85 Great Portland Street, First Floor, London, W1W 7LT	2	0.2%
3 Crewe Road, Sandbach, Cheshire, CW11 4NE	2	0.2%
130, Old Street, London, EC1V 9BD	2	0.2%
Other values (958)	961	96.1%

Length

Histogram of lengths of the category

Value	Count	Frequency (%)
road	350	4.4%
london	225	2.8%
street	161	2.0%
house	150	1.9%
park	68	0.9%
lane	62	0.8%
west	53	0.7%
the	52	0.7%
avenue	45	0.6%
1	44	0.6%
Other values (3636)	6745	84.8%

Most occurring characters

Value	Count	Frequency (%)
	6957	13.7%
e	3485	6.9%
,	3257	6.4%
o	2823	5.6%
r	2402	4.7%
n	2269	4.5%
a	2224	4.4%
t	1920	3.8%
s	1565	3.1%
d	1491	2.9%
Other values (64)	22302	44.0%

Most occurring categories

Value	Count	Frequency (%)
Lowercase Letter	26850	53.0%
Uppercase Letter	9036	17.8%
Space Separator	6957	13.7%
Decimal Number	4407	8.7%
Other Punctuation	3321	6.6%
Dash Punctuation	116	0.2%
Open Punctuation	4	< 0.1%
Close Punctuation	4	< 0.1%

Most frequent character per category

Lowercase Letter

Value	Count	Frequency (%)
e	3485	13.0%
o	2823	10.5%
r	2402	8.9%
n	2269	8.5%
a	2224	8.3%
t	1920	7.2%
s	1565	5.8%
d	1491	5.6%
i	1459	5.4%
l	1449	5.4%
Other values (16)	5763	21.5%

Uppercase Letter

Value	Count	Frequency (%)
S	836	9.3%
L	693	7.7%
R	674	7.5%
B	644	7.1%
H	614	6.8%
C	606	6.7%
W	504	5.6%
A	456	5.0%
E	429	4.7%
N	416	4.6%
Other values (16)	3164	35.0%

Decimal Number

Value	Count	Frequency (%)
1	996	22.6%
2	649	14.7%
3	464	10.5%
4	395	9.0%
7	361	8.2%
5	360	8.2%
6	349	7.9%
0	294	6.7%
8	281	6.4%
9	258	5.9%

Other Punctuation

Value	Count	Frequency (%)
,	3257	98.1%
/	24	0.7%
.	24	0.7%
&	8	0.2%
'	5	0.2%
:	3	0.1%

Open Punctuation

Value	Count	Frequency (%)
(	3	75.0%
[	1	25.0%

Close Punctuation

Value	Count	Frequency (%)
)	3	75.0%
]	1	25.0%

Space Separator

Value	Count	Frequency (%)
	6957	100.0%

Dash Punctuation

Value	Count	Frequency (%)
-	116	100.0%

Most occurring scripts

Value	Count	Frequency (%)
Latin	35886	70.8%
Common	14809	29.2%

Most frequent character per script

Latin

Value	Count	Frequency (%)
e	3485	9.7%
o	2823	7.9%
r	2402	6.7%
n	2269	6.3%
a	2224	6.2%
t	1920	5.4%
s	1565	4.4%
d	1491	4.2%
i	1459	4.1%
l	1449	4.0%
Other values (42)	14799	41.2%

Common

Value	Count	Frequency (%)
	6957	47.0%
,	3257	22.0%
1	996	6.7%
2	649	4.4%
3	464	3.1%
4	395	2.7%
7	361	2.4%
5	360	2.4%
6	349	2.4%
0	294	2.0%
Other values (12)	727	4.9%

Most occurring blocks

Value	Count	Frequency (%)
ASCII	50695	100.0%

Most frequent character per block

ASCII

Value	Count	Frequency (%)
	6957	13.7%
e	3485	6.9%
,	3257	6.4%
o	2823	5.6%
r	2402	4.7%
n	2269	4.5%
a	2224	4.4%
t	1920	3.8%
s	1565	3.1%
d	1491	2.9%
Other values (64)	22302	44.0%

df_index
statementID

statementID
df_index

statementID
df_index

Spearman's ρ

The Spearman's rank correlation coefficient (ρ) is a measure of monotonic correlation between two variables, and is therefore better in catching nonlinear monotonic correlations than Pearson's r. It's value lies between -1 and +1, -1 indicating total negative monotonic correlation, 0 indicating no monotonic correlation and 1 indicating total positive monotonic correlation.

To calculate ρ for two variables X and Y, one divides the covariance of the rank variables of X and Y by the product of their standard deviations.

Pearson's r

The Pearson's correlation coefficient (r) is a measure of linear correlation between two variables. It's value lies between -1 and +1, -1 indicating total negative linear correlation, 0 indicating no linear correlation and 1 indicating total positive linear correlation. Furthermore, r is invariant under separate changes in location and scale of the two variables, implying that for a linear function the angle to the x-axis does not affect r.

To calculate r for two variables X and Y, one divides the covariance of X and Y by the product of their standard deviations.

Kendall's τ

Similarly to Spearman's rank correlation coefficient, the Kendall rank correlation coefficient (τ) measures ordinal association between two variables. It's value lies between -1 and +1, -1 indicating total negative correlation, 0 indicating no correlation and 1 indicating total positive correlation.

To calculate τ for two variables X and Y, one determines the number of concordant and discordant pairs of observations. τ is given by the number of concordant pairs minus the discordant pairs divided by the total number of pairs.

Phik (φk)

Phik (φk) is a new and practical correlation coefficient that works consistently between categorical, ordinal and interval variables, captures non-linear dependency and reverts to the Pearson correlation coefficient in case of a bivariate normal input distribution. There is extensive documentation available here.

Count
Matrix

A simple visualization of nullity by column.

Nullity matrix is a data-dense display which lets you quickly visually pick out patterns in data completion.

First rows

	df_index	statementID	address
0	9699	9120979901932274174	113, Ellisland, Kirkintilloch, Scotland, G66 2UA
1	8626	16443079243458318158	2, Queens Quay, Belfast, Co Antrim, BT3 9QQ
2	8713	12336355866542270999	Redbridge Farm, Dolmans Hill, Lytchett Matravers, Poole, Dorset, BH16 6HP
3	5913	13914046920756315847	Churton House Chester Road, Churton, Chester, CH3 6LA
4	10359	4149813677626374910	122 Sunnymead, Peterborough, PE4 5BZ
5	3712	2427637428553505209	Pied House Church Road, Twinstead, Sudbury, Suffolk, CO10 7NA
6	10151	13841278220370464048	71-75 Shelton Street, London, Greater London, WC2H 9JQ
7	4089	3378778897477776452	3 Caroline Court, 13 Caroline Street, Birmingham, B3 1TR
8	6029	17069500166715706930	161 High Street, Barnet, EN5 5SU
9	11911	6796589355598666233	Unit D11 Rivington Court, Walter Leigh Way, Moss Industrial Estate, Leigh, WN7 3PT

Last rows

	df_index	statementID	address
990	7831	11058219476647133923	555-557, Cranbrook Road, Ilford, London, IG2 6HE
991	463	843042277929823282	114 Lebanon Road, Croydon, CR0 6US
992	2690	7038664883610238888	20-22 Wenlock Road, London, N1 7GU
993	1917	1287251194342704153	22 Clare Gardens, Barking, IG11 9JH
994	12590	5898018506881324871	7, Highbury Hill, London, N5 1SU
995	12470	9448672836645524891	White Cross Business Park, South Road, Lancaster, Lancashire, LA1 4XQ
996	7754	15802196399375077567	Wallhouse, Mansion, Torpichen, Bathgate, Westlothian, EN48 4NQ
997	9203	1224164980050073317	Marquis House, 54 Richmond Road, Twickenham, Middlesex, TW1 3BE
998	6403	3967766129471749989	50 Belsize Park, London, NW3 4EE
999	7089	10922022144558512691	28, Dunston Hill, Tring, HP23 4AT

Overview

Variables

Common Values

Length

Most occurring characters

Most occurring categories

Most frequent character per category

Lowercase Letter

Uppercase Letter

Decimal Number

Other Punctuation

Open Punctuation

Close Punctuation

Space Separator

Dash Punctuation

Most occurring scripts

Most frequent character per script

Latin

Common

Most occurring blocks

Most frequent character per block

ASCII

Interactions

Correlations

Spearman's ρ

Pearson's r

Kendall's τ

Phik (φk)

Missing values

Sample

First rows

Last rows